1. Technical Field
The present invention relates to image encoding devices, image decoding devices and integrated circuits used therein, which are characterized by transforms and inverse transforms between a time domain and a frequency domain.
2. Background Art
As a moving image encoding method, MPEG-2 has already been used for digital broadcasting and DVDs, and MPEG-4 has been used in the fields of Internet streaming and mobile communications. A more compression-effective encoding method is standardized by the JVT (Joint Video Team). This encoding method is known as “H.264/MPEG-4 AVC” or “MPEG-4 part 10” (hereinafter, referred to as “H.264”).
In methods, such as MPEG-2 and MPEG-4, which perform compression (encoding) based on intra-screen correlations, an 8×8 pixel block is first subjected to a discrete cosine transform (DCT) from a time domain to a frequency domain. Then, each coefficient for the obtained frequency domain is divided by a divisor (a quantization step), and the remainder is rounded. This achieves encoding and compression. In a decoding process, conversely, inverse quantization is performed by multiplying an 8×8 pixel block by a quantization step. After the inverse quantization, an inverse discrete cosine transform (IDCT) is performed for transformation from the frequency domain to the time domain.
On the other hand, in H.264, a macroblock consisting of 16×16 pixels is divided into 4×4 pixel blocks at the time of encoding. The obtained 4×4 blocks are each taken as a unit, and transformed from the time domain to the frequency domain by bit shifting, addition and subtraction. At the time of decoding, conversely, the 4×4 blocks obtained as the frequency domain are transformed from the frequency domain to the time domain by bit shifting, addition and subtraction. Such a transform from the time domain to the frequency domain by bit shifting, addition and subtraction is referred to as the “integer transform”. Also, a transform from the frequency domain to the time domain by bit shifting, addition and subtraction is referred to as the “inverse integer transform” (see Non-Patent Publication 1).
Further, in H.264, when transforming from the time domain to the frequency domain for intra-frame encoding in units of 16×16 pixels (hereinafter, referred to as the “Intra 16×16 mode”) and color difference encoding, an integer transform is performed, and thereafter, DC components obtained by the integer transform are subjected to a Hadamard transform (see Non-Patent Publication 1) FIG. 30 is a diagram illustrating the flow of transforming pixel data in the Intra 16×16 mode. First, 16×16 pixels having a brightness in the time domain are subjected to the integer transform to the frequency domain in units of 4×4 pixels. Then, a 4×4 matrix, which is obtained by extracting DC components included in the integer-transformed 4×4 pixel units, is compressed by the Hadamard transform. Thereafter, the DC components compressed by the Hadamard transform are quantized, and integer-transformed data other than the DC components is quantized. This realizes encoding in the Intra 16×16 mode.
FIG. 31 is a flowchart illustrating the procedure for a decoding process in H.264. First, whether encoded data is Intra 16×16 mode-encoded data, color difference data or other data is determined (step S901). If encoded by the Intra 16×16 mode, DC components of the encoded data are subjected to a Hadamard transform (step S902), and the Hadamard-transformed DC components and other components are inversely quantized (step S903). If the encoded data is color difference data, DC components of the encoded data are subjected to a Hadamard transform (step S904), and the Hadamard-transformed DC components and other components are inversely quantized (step S905). If the encoded data is other data, it is inversely quantized (step S906). Thereafter, the inversely quantized data is decoded by an inverse integer transform (step S907).
Expression 1 is a transform expression for subjecting a 4×4 matrix C(cij) (i, j=1, 1, 2, 3) consisting only of DC components to the Hadamard transform at steps S902 and S904. In Expression 1, the 4×4 matrix C(Cij) is multiplied by a Hadamard transform matrix A from the right and left sides to obtain a Hadamard-transformed 4×4 matrix f (see Non-Patent Publication 1, page 131, expression (8-254)).
                    f        =                  ACA          =                                                                                   [                                                                                    1                                                                    1                                                                    1                                                                    1                                                                                                            1                                                                    1                                                                                              -                          1                                                                                                                      -                          1                                                                                                                                    1                                                                                              -                          1                                                                                                                      -                          1                                                                                            1                                                                                                            1                                                                                              -                          1                                                                                            1                                                                                              -                          1                                                                                                      ]                                [                                                                  ⁢                                                                                                    C                        00                                                                                                            C                        01                                                                                                            C                        02                                                                                                            C                        03                                                                                                                                                C                        10                                                                                                            C                        11                                                                                                            C                        12                                                                                                            C                        13                                                                                                                                                C                        20                                                                                                            C                        21                                                                                                            C                        22                                                                                                            C                        23                                                                                                                                                C                        30                                                                                                            C                        31                                                                                                            C                        32                                                                                                            C                        33                                                                                            ]                            ⁢                                                          [                                                          ⁢                                                                    1                                                        1                                                        1                                                        1                                                                                        1                                                        1                                                                              -                      1                                                                                                  -                      1                                                                                                            1                                                                              -                      1                                                                                                  -                      1                                                                            1                                                                                        1                                                                              -                      1                                                                            1                                                                              -                      1                                                                                  ⁢                                                          ]                                                          [                  Expression          ⁢                                          ⁢          1                ]            
Expression 2 indicates an arithmetic expression for subjecting frequency-domain components dij (i, j=0, 1, 2, 3) to an inverse integer transform to obtain time-domain components fij (i, j=0, 1, 2, 3) at step S907. The arithmetic expression indicated in Expression 2 is used to first obtain components ei0 ei3 and components fi0 to fi3 for an arbitrary i (=0, 1, 2, 3) in the frequency domain (see Non-Patent Publication 1, page 133, expressions (8-266) to (8-273)). Further, the components fij are transposed (i and j are interchanged), and the components obtained by the transposition are taken as components dij to obtain the components ei0 to ei3 and the components fi0 to fi3 again. The ultimately obtained components fij are time-domain components obtained by the inverse integer transform. Note that in Expression 2, (x>>k) is meant to indicate that x is shifted to the right by k bits (the same below). In addition, (x<<k) is meant to indicate that x is shifted to the left by k bits (the same below).
                                                                                          e                                      i                    ⁢                                                                                  ⁢                    0                                                  =                                                      d                                          i                      ⁢                                                                                          ⁢                      0                                                        +                                      d                                          i                      ⁢                                                                                          ⁢                      2                                                                                                                                                                e                                      i                    ⁢                                                                                  ⁢                    1                                                  =                                                      d                                          i                      ⁢                                                                                          ⁢                      0                                                        -                                      d                                          i                      ⁢                                                                                          ⁢                      2                                                                                                                                                                e                                      i                    ⁢                                                                                  ⁢                    2                                                  =                                                      (                                                                  d                                                  i                          ⁢                                                                                                          ⁢                          1                                                                    ⪢                      1                                        )                                    -                                      d                                          i                      ⁢                                                                                          ⁢                      3                                                                                                                                                                e                                      i                    ⁢                                                                                  ⁢                    3                                                  =                                                      d                                          i                      ⁢                                                                                          ⁢                      1                                                        +                                      (                                                                  d                                                  i                          ⁢                                                                                                          ⁢                          3                                                                    ⪢                      1                                        )                                                                                                                                            f                                      i                    ⁢                                                                                  ⁢                    0                                                  =                                                      e                                          i                      ⁢                                                                                          ⁢                      0                                                        +                                      e                                          i                      ⁢                                                                                          ⁢                      3                                                                                                                                                                f                                      i                    ⁢                                                                                  ⁢                    1                                                  =                                                      e                                          i                      ⁢                                                                                          ⁢                      1                                                        +                                      e                                          i                      ⁢                                                                                          ⁢                      2                                                                                                                                                                f                                      i                    ⁢                                                                                  ⁢                    2                                                  =                                                      e                                          i                      ⁢                                                                                          ⁢                      1                                                        -                                      e                                          i                      ⁢                                                                                          ⁢                      2                                                                                                                                                                f                                      i                    ⁢                                                                                  ⁢                    3                                                  =                                                      e                                          i                      ⁢                                                                                          ⁢                      0                                                        -                                      e                                          i                      ⁢                                                                                          ⁢                      3                                                                                                          ❘                            [                  Expression          ⁢                                          ⁢          2                ]            
Further, it has been proposed to adaptively change the size of blocks to be processed in a macroblock consisting of 16×16 pixels, and subject the blocks with a changed size to an integer transform (see Non-Patent Publication 2). Expression 3 indicates a transform expression for an inverse integer transform in the case where the size of blocks to be processed is 8×8 (see Non-Patent Publication 2, page 142, expressions (12-10) and (12-12)). In Expression 3, an 8×8 matrix W(wij) (i, j=0, 1, . . . 7) represents a frequency domain for the inverse integer transform. An 8×8 matrix T is a matrix for the inverse integer transform. An 8×8 matrix X represents a time domain after the inverse integer transform. The transform expression indicated in Expression 3 makes it possible to obtain the time domain after the inverse integer transform.
                                              ⁢                              X            =                                          T                T                            ⁢              WT              ⁢                                                          ⁢                              (                                                      T                    T                                    ⁢                                                                          ⁢                  is                  ⁢                                                                          ⁢                  a                  ⁢                                                                          ⁢                  transposed                  ⁢                                                                          ⁢                  matrix                                )                                              ⁢                                          ⁢                                          ⁢                      W            =                          [                                                                                          W                      00                                                                            …                                                                              W                      07                                                                                                            ⋮                                                        ⋱                                                        ⋮                                                                                                              W                      70                                                                            …                                                                              W                      77                                                                                  ]                                ⁢                                          ⁢                      T            =                          [                                                                    13                                                        13                                                        13                                                        13                                                        13                                                        13                                                        13                                                        13                                                                                        19                                                        15                                                        9                                                        3                                                                              -                      3                                                                                                  -                      9                                                                                                  -                      15                                                                                                  -                      19                                                                                                            17                                                        7                                                                              -                      7                                                                                                  -                      17                                                                                                  -                      17                                                                                                  -                      7                                                                            7                                                        17                                                                                        9                                                        3                                                                              -                      19                                                                                                  -                      15                                                                            15                                                        19                                                                              -                      3                                                                                                  -                      9                                                                                                            13                                                                              -                      13                                                                                                  -                      13                                                                            13                                                        13                                                                              -                      13                                                                                                  -                      13                                                                            13                                                                                        15                                                                              -                      19                                                                                                  -                      3                                                                            9                                                                              -                      9                                                                            3                                                        19                                                                              -                      15                                                                                                            7                                                                              -                      17                                                                            17                                                                              -                      7                                                                                                  -                      7                                                                            17                                                                              -                      17                                                                            7                                                                                        3                                                                              -                      9                                                                            15                                                                              -                      19                                                                            19                                                                              -                      15                                                                            9                                                                              -                      3                                                                                  ]                                                          [                  Expression          ⁢                                          ⁢          3                ]            
A matrix Ta for an inverse integer transform as shown in Expression 4 is proposed as a replacement for the matrix T indicated in Expression 3 (see Non-Patent Publication 3, pages 165 to 166, expressions (8-341) to (8-356)).
                    Ta        =                              [                                                            8                                                  8                                                  8                                                  8                                                  8                                                  8                                                  8                                                  8                                                                              12                                                  10                                                  6                                                  3                                                                      -                    3                                                                                        -                    6                                                                                        -                    10                                                                                        -                    12                                                                                                8                                                  4                                                                      -                    4                                                                                        -                    8                                                                                        -                    8                                                                                        -                    4                                                                    4                                                  8                                                                              10                                                                      -                    3                                                                                        -                    12                                                                                        -                    6                                                                    6                                                  12                                                  3                                                                      -                    10                                                                                                8                                                                      -                    8                                                                                        -                    8                                                                    8                                                  8                                                                      -                    8                                                                                        -                    8                                                                    8                                                                              6                                                                      -                    12                                                                    3                                                  10                                                                      -                    10                                                                                        -                    3                                                                    12                                                                      -                    6                                                                                                4                                                                      -                    8                                                                    8                                                                      -                    4                                                                                        -                    4                                                                    8                                                                      -                    8                                                                    4                                                                              3                                                                      -                    6                                                                    10                                                                      -                    12                                                                    12                                                                      -                    10                                                                    6                                                                      -                    3                                                                        ]                    ·                                          ⁢                      1            /                                                  ⁢            8                                              [                  Expression          ⁢                                          ⁢          4                ]            
Expression 5 indicates an arithmetic expression for performing an inverse integer transform using the matrix Ta indicated in Expression 4. In Expression 5, in[j] represents wij in the case where an arbitrary i (i=0, 1, . . . 7) is selected as a row. When the arbitrary i is selected as a row, it is possible to calculate a[0] to a[7] and b[0] to b[7] by the arithmetic expression in Expression 5. By using b[0] to b[7], out[0] to out[7] for the arbitrary i are calculated. The obtained out[0] to out[7] compose a matrix WTa for the arbitrary i. Next, i and j in out[0] to out[7] are transposed, and the transposed values are taken as in[0] to in[7]. Then, Expression 5 is similarly used for an arithmetic operation to obtain out[0] to out[7]. The obtained out [0] to out [7] compose a matrix TaTWTa, and therefore represent a time domain after the inverse integer transform.a[0]=in[0]+in[4]a[4]=in[0]−in[4]a[2]=(in[2]>>1)−in[6]a[6]=in[2]+(in[6]>>1)b[0]=a[0]+a[6]b[2]=a[4]+a[2]b[4]=a[4]−a[2]b[6]=a[0]−a[6]a[1]=−in[3]+in[5]−in[7]−(in[7]>>1)a[3]=in[1]+in[7]−in[3]−(in[3]>>1)a[5]=−in[1]+in[7]+in[5]+(in[5]>>1)a[7]=in[3]+in[5]+in[1]+(in[1]>>1)b[1]=a[1]+(a[7]>>2)b[7]=−a[1]>>2+a[7]b[3]=a[3]+(a[5]>>2)b[5]=(a[3]>>2)−a[5]out[0]=b[0]+b[7]out[1]=b[2]+b[5]out[2]=b[4]+b[3]out[3]=b[6]+b[1]out[4]=b[6]−b[1]out[5]=b[4]−b[3]out[6]=b[2]−b[5]out[7]=b[0]−b[7]  [Expression 5][Non-Patent Publication 1] Draft ITU-T Recommendation and Final Draft International Standard of Joint Video Specification (ITU-T Rec. H.264|ISO/IEC 14496-10 AVC), Doc. JVT-G050r1, May 27, 2003, pages 127 to 134[Non-Patent Publication 2] Joint Final Committee Draft (JFCD) of Joint Video Specification (ITU-T Rec. H.264|ISO/IEC 14496-10 AVC), Doc. JVT-D157, Aug. 10, 2002, pages 141 to 143[Non-Patent Publication 3] Draft of Version 4 of H.264/AVC (ITU-T Recommendation H.264 and ISO/IEC 14496-10 (MPEG-4 part 10) Advanced Video Coding), Doc. JVT-N050d1, Jan. 28, 2005, pages 153 to 170
For decoding in H.264 as described above, it is necessary to use the Hadamard transform, the 4×4 inverse integer transform, and the 8×8 inverse integer transform in combination. If separate arithmetic units respectively corresponding to the Hadamard transform, the 4×4 inverse integer transform, and the 8×8 inverse integer transform are prepared, the circuit scale of the decoding device is increased. The same can be said of the encoding device.
Also, in recent years, a plurality of standards for image encoding coexist, and new standards are further proposed one after another. In such circumstances, it is necessary to provide an image encoding device and an image decoding device, which are capable of readily dealing with a plurality of standards as well as standards that will be introduced anew. Naturally, arithmetic units respectively corresponding to the individual standards can be separately prepared and provided in the decoding device and the encoding device, though this increases the circuit scale.
Therefore, an object of the present invention is to provide an image encoding device and an image decoding device, which are capable of dealing with various encoding standards, while achieving compactness.